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TWO RESULTS ON ENTROPY, CHAOS, AND INDEPENDENCE 
IN SYMBOLIC DYNAMICS 

FRYDERYK FALNIOWSKI, MARCIN KULCZYCKI, DOMINIK KWIETNIAK, AND JIAN LI 


Abstract. We survey the connections between entropy, chaos, and indepen¬ 
dence in topological dynamics. We present extensions of two classical results 
placing the following notions in the context of symbolic dynamics: 

(1) Equivalence of positive entropy and the existence of a large (in terms of 
asymptotic and Shnirelman densities) set of combinatorial independence 
for shift spaces. 

(2) Existence of a mixing shift space with a dense set of periodic points with 
topological entropy zero and without ergodic measure with full support, 
nor any distributionally chaotic pair. 

Our proofs are new and yield conclusions stronger than what was known before. 


1. Introduction. 

Furstenberg nu p. 38] calls a dynamical system deterministic if its topological 
entropy vanishes. One may argue that the future of a deterministic dynamical 
system can be predicted if its past is known (see [401 Chapter 7]). In a similar way 
positive entropy may be related to randomness and chaos. 

This article surveys the relations between entropy, independence, and chaos, 
adding a new twist to two classical results. 

The first result describes in what sense positive entropy may be understood as 
independence. It turns out that for a dynamical system there is a deep connection 
between positive entropy and randomness defined through the notion of the combi¬ 
natorial independence. General results of this kind can be found in [20] . A special 
instance of this result (see EDI Theorem 8.1]) says that a symbolic dynamical sys¬ 
tem (a shift space) A C {0,1} 11 has positive entropy if and only if there is a large set 
of indices along which points from X behave like {0, l}-valued independent random 
variables. We strengthen this theorem by proving that positive topological entropy 
of a shift space is equivalent to the existence of an independence set for which as¬ 
ymptotic and Shnirelman densities are equal and positive. This shows that, for a 
shift space with positive entropy, one can find an independence set which is large 
and structured. One can easily adapt our reasoning to the general context. 

Before we describe our second result, let us consider the following properties that 
a dynamical system (A, T) may have. 

(1) There exists an ergodic T-invariant Borel probability measure on X with 
full support (that is, an invariant measure positive on all nonempty open 
subsets of A). 
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(2) The system ( X , T ) is topologically transitive, that is, for any pair of nonempty 
open sets U, V C X there exists a positive integer n with T~ n (U ) fl V ^ 0. 

(3) There exists a T-invariant (not necessarily ergodic) Borel probability mea¬ 
sure on X with full support. 

It is straightforward that m implies (J2]) and, trivially, (0. It is also relatively easy 
to prove that © does not imply © and hence neither implies ©. According to ©5] , 
H. Keynes asked whether © and © together imply ©. B. Weiss answered this 
question negatively in ©1. He constructed a transitive shift space with a dense 
set of periodic points and no ergodic invariant measure of full support. He also 
noted, omitting the proof, that his example is in fact topologically mixing and has 
zero topological entropy. Furthermore, Weiss conjectured [39] Remark 1] that the 
only ergodic measures for his system are those concentrated on the orbit of a single 
periodic point. Note that Huang and Ye constructed a uniformly positive entropy 
system without an ergodic invariant measure of full support (see [T7l. Theorem 9.6]). 

We provide an alternative construction of a similar example. Our method allows 
us to prove that the system we have defined possesses all the properties proved 
or conjectured about the example from [391 . Furthermore, we show that our ex¬ 
ample does not possess any distributionally chaotic pairs. This extends a result 
by Oprocha [28] who sketched a construction of a Devaney chaotic system without 
any DC1 distributionally chaotic pairs. In addition, we show that our example is 
w-chaotic, but not w*-chaotic. To do this we provide a new method of constructing 
w-chaotic sets. 


2. Notation and definitions. 

Let N denote the set of positive integers. We denote the number of elements 
of a finite set A by |A|. Given any real number x, we write f° r the largest 
integer not greater than x. A sequence of real numbers {an}^^ is subadditive if 
ctm+n < (t m + <i n for all ui, n € N. We recall that Fekete’s Lemma states that 
if a sequence { a n of nonnegative numbers is subadditive, then the sequence 
{^ a n}^Li converges to a limit equal to the infimum of the terms of this sequence. 

A (topological) dynamical system is a pair (X,T), where X is a compact metric 
space and T: X —» X is a continuous map. By p we denote a compatible metric for 
X. A dynamical system ( X , T ) is transitive if for any nonempty open sets U, V C X 
there is n £ N such that T~ n {V)nU ^ 0. We say that {X, T) is topologically mixing 
if for any nonempty open sets U, V C X there is N € N such that T~ n (V ) fl U ^ 0 
for all n > N. The set of limit points of the sequence {T n (*)}„ e N is called the 
Lo-limit set of x and denoted by ujt(x). A set M C X is called minimal if it is 
nonempty, closed, invariant, and contains no proper subset with these properties. 

We assume the reader is familiar with the basic notions of topological dynamics 
and ergodic theory (see [35]). In particular, we assume that the reader knows the 
definition of topological entropy. 


3. Definitions of chaos and their relations with topological entropy 

In this section we collect some definitions of chaos and survey connections be¬ 
tween them and topological entropy. 
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3.1. Li-Yorke chaos. Let (A, T) be a dynamical system and (x,y) £ X x X. We 
say that (x,y) is a Li-Yorke pahtj if 

\immf p(T n (x),T n (y)) = 0, 

n—> oo 

limsup p(T n (x),T n (y)) > 0. 

n—¥ oo 

A dynamical system (X,T) is Li-Yorke chaotic if there is an uncountable set S C X 
such that every pair (x, y) with x,y £ S and x ^ y is a Li-Yorke pair. A nice survey 
of properties of Li-Yorke chaotic systems can be found in [8j . 

3.2. Distributional chaos. Given x,y £ X and t £ R we define an upper and 
lower distribution function by 

F xy (t) = liminf- |{0 < j < n - 1 : p{T J (x),T 3 (y)) < t}| , 

n—too n 

F* y [t) = limsup — |{0 < j < n — 1 : p{T^(x),T\y)) < t}| . 

n—»oo Tl 

Clearly, t i-A F xy (t ) and t F* (t ) are nondecreasing, 0 < F xy (t) < F* (t) < 1 
for all t £ R, F xy (t ) = F* y (t) = 0 for all t < 0, and F xy (t ) = F* y (t) = 1 for all 
t > diamX. Distributional chaos was introduced in [35]. Following [3] we say that 
a pair (x, y) £ X x X is a DC1 -pair if F* (t) = 1 for all t > 0, and F xy {s) = 0 for 
some s > 0. A pair (x, y) is a DC2 -pair if F* (t ) = 1 for all t > 0, and F xy (s ) < 1 for 
some s > 0. Finally, a DC3- pair is a pair (x, y) £ X x X such that F xy (t ) < F* (t) 
for all t. in some interval of positive length. 

Let i £ {1,2,3}. The dynamical system (A, T) is distributionally chaotic of type 
i (or DC i-chaotic for short), if there is an uncountable set S C X such that any 
pair of distinct points from S' is a DCApair. 

3.3. Devaney chaos. Devaney calls a dynamical system (X,T) chaotic if T is 
transitive, T-periodic points are dense in X , and T has sensitive dependence on 
initial conditions (see 0)- It turns out that if X is infinite, then the sensitive 
dependence follows from the other two conditions (see [H [H]). Since we restrict 
our attention to compact metric spaces without isolated points we say that (X,T) 
is Devaney chaotic if T is transitive and T-periodic points are dense in X. Such 
systems are also known as P-systems (see fl4]h 

3.4. w-chaos. We say that a dynamical system (A, T) is io-chaotic if there exists 
an uncountable set S C X such that for any x,y £ S with i/j/we have 

• ujT(x)\uiT(y) is uncountable, 

• ujt(x) r\Lo T (y) is nonempty, 

• ujt(x) is not contained in the set of periodic points. 

Li |24] introduced w-chaos and proved that it is equivalent to positive topological 
entropy for interval maps. In [25] Li defined a variant of w-chaos, called to*-chaos. 
A dynamical system (A, T) is w* -chaotic if there exists an uncountable set S C A 
such that for any x 7 y £ S with x ^ y we have 

• wt(i)\wt(!/) contains an infinite minimal set, 

• ujt(x) n u T {y) is nonempty. 


^Such pairs appeared in the seminal paper of Li and Yorke m- 
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Since an infinite minimal set is uncountable and does not contain a periodic point, 
uj* implies w-chaos. All examples of w-chaotic maps known to us are in fact ui*- 
chaotic. As is common for various notions of chaos in a general setting of an 
arbitrary compact metric space, w-chaos is independent of most other notions of 
chaos. 

3.5. Entropy vs. chaos. There is no connection between positive topological 
entropy and Devaney chaos. It is known that Devaney chaos implies Li-Yorke 
chaos [I2| and does not imply distributional chaos [55] . The example in [55] shows 
that Devaney chaos (hence Li-Yorke chaos as well by [16]) does not imply positive 
entropy. In general, positive entropy neither implies topological transitivity nor 
existence of periodic points. Nevertheless, Li [24 proved that for maps on the 
interval positive topological entropy is equivalent to the existence of a subsystem 
chaotic in the sense of Devaney. The question whether positive topological entropy 
implies Li-Yorke chaos or DC2 distributional chaos remained open for some time, 
but eventually both implications turned out to be true; see [6] and [10], respectively. 
The consequences of positive topological entropy for dynamics of pairs and tuples 
were also examined in HI EH]- Distributional chaos of type 2 implies Li-Yorke chaos 
by definition. The converse implication is not true, because if X = [0,1], then 
by [55] distributional chaos DC3 is equivalent to the positive topological entropy, 
and there are Li-Yorke chaotic interval maps with zero topological entropy (as was 
shown independently by Smftal m and Xiong [44]). Pikula proved that there is no 
connection between positive topological entropy and DC1 distributional chaos [32], 
therefore Li-Yorke chaos doesn’t imply positive topological entropy, either. Pikula 
[321 also constructed an example of w-chaotic dynamical system without Li-Yorke 
pairs. Since there exist minimal systems with positive topological entropy, none 
of the following properties: positive topological entropy, distributional chaos, Li- 
Yorke chaos imply w-chaos (just note that if (X,T) is minimal, then u>t{x) = X 
for every x £ X). Downarowicz and Ye [11] constructed a transitive dynamical 
system (X, T) which is Devaney chaotic and every point x £ X is either transitive 
(wr(x) = X) or periodic. Such a system cannot be w-chaotic. 

All above is only a glimpse of the vast literature of the subject. For deeper 
discussion of these matters we refer the reader to the excellent surveys by Blanchard 
0, Glasner and Ye Q3], Li and Ye [23l . 


4. Symbolic dynamics 


With regard to symbolic dynamics we follow the notation and terminology of 
Lind and Marcus [27] as closely as possible, except that we consider only one-sided 
shifts. 


Definitions (Full shifts). Let A be a nonempty finite set. We call A the alphabet 
and elements of A are symbols. The full A-shift is the set 

A n = (a: = {xi} c *L 1 : Xi £ A for all i £ N}. 

We equip A with the discrete topology and A N with the product topology. We 
usually write an element of A N as x = = X 1 X 2 X 3 .... Then A N is a compact 

topological space and the formula 


( 1 ) 


p{x,y) 


0, if x = y, 

2~min{jeN:x j jty j } ^ x ^ 
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defines a metric on A N which is compatible with the product topology. The shift 
map a : A N —> A N is a continuous transformation given by 

x = {Xi }™i ct(x) = {x i+ i}£i. 

That is, er(x) is the sequence obtained by dropping the first symbol of x. A full 
r-shift is the full shift over the alphabet {0,1,..., r — 1} and a full binary shift is 
the full 2-shift. 

Definitions (Blocks). A block over A is a finite sequence of symbols and its length 
is the number of its symbols. An n-block stands for a block of length n. The empty 
block, denoted by e, is the unique block with no symbols and length zero. The 
set of all blocks over A (including e) is denoted by A*. The concatenation of two 
blocks u = a\... ak and v = b\ ... bi is the block uv = a±... akb\ .. .bi. We write 
u n for the concatenation of n > 1 copies of a block u and u°° for the sequence 
uuu ... £ A N . 

By xuj] we denote the block XjXi+i ... Xj, where 1 < i < j and x = (xi)f2. 1 £ A N . 
We say that a block w £ A* occurs in x and x contains w if w = Xu j] for some 
integers 1 < i < j. Note that e occurs in every sequence from A N . Similarly, given 
an n-block w = w\... w n £ A* we define w uji = WiWi+i .. .Wj £ A* for each 
1 < i < j < n. A prefix of a block z £ A* is any u such that z = uv for some 
v £ A*. 

Definitions (Shift spaces). Given any collection T of blocks over A (i.e., a subset of 
A*) we define a shift space specified by T, denoted by Ay, as the set of all sequences 
from A n which do not contain any blocks from r J. We say that T is a collection of 
forbidden blocks for Xj. 

A shift space is a set X C A N such that X = Xj for some ? C A*. A binary 
shift space is a shift space over the alphabet {0,1}. Equivalently, X C A N is a shift 
space if it is a closed and a-invariant subset of C A N . 

Definition (Language of a shift space). The language of a shift space X over A is 
the set of blocks over A which do occur in some sequence x £ X. We denote it by 
'B(X). We write r B n (X) for the set of all n-blocks contained in 23 (X). Similarly, 
23 (x) C A* is the collection of all blocks occurring in x £ A N . The language of a 
shift space determines the shift space: x £ A N belongs to a shift space X if and 
only if for every k £ N the initial block xmm is in 23(X). 

A cylinder of an n-word w £ A* in a shift space X is the set 
Hx — {x £ X : X[i, n ] = mj}. 

If a space X is clear from the context, we call [uiJjv the cylinder of w. The collection 
of all cylinders {[w]x : w £ 23(A)} is a basis of the topology of A. If £ is a language 
of some shift space over A, then C is factorial, meaning that if u £ C and u = vw 
for some blocks v,w £ A*, then both v and w also belong to C, and prolongable, 
meaning that for every block u in C there is a symbol a £ A such that ua also 
belongs to C. 

Conversely, every factorial and prolongable subset C C A* determines a shift 
space Xc such that C is the language of A (see j27j Proposition 1.3.4]). A collection 
of forbidden blocks defining A is T = A* \ C. A point x £ A N is in A c if and only 
if x\i,j] £ £ for all i, j £ N with i < j. 
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We will use superscripts in brackets to denote indices for sequences of blocks in 
A* or points in Jl N . That is, we write {«/"'})£!-! for a sequence of blocks and we 
use subscripts for enumerating symbols in the block: = w[.. -w ^. 

In its full generality, the concept of entropy was defined by Adler, Konheim and 
McAndrew [1] for any continuous map T: X — > X on an arbitrary compact topo¬ 
logical space X. The definition below, specific to symbolic dynamics, is equivalent 
to the general one. 

Definition (Entropy). Let X C be a nonempty shift space and let m,n £ N. 
Observe that \‘B nl+n (X)\ < \‘S m (X)\ ■ |ff„(X)|, and hence 

log |® m+n (X)| < log |® m (X)| + log |®„(X)|. 

Using Fekete’s lemma we define the entropy of X, denoted by h(X), as 

h(X) = lim — log|23„(X)| = inf - log |B„(X)|. 

n—¥ oo n n> 1 n 

5. Entropy and independence in symbolic dynamics 

Independence is certainly among the most popular terms used to single out an 
interesting mathematical phenomenon. Consider the full shift over {0,1}. Clearly, 
the full shift is as random as possible - it is a model containing all possible out¬ 
comes of an infinite sequence of fair coin tosses (we set Xi = 1 if the coin turns heads 
on the i-th toss, and Xi = 0 otherwise). Anything that can happen is encoded in 
some point x — more precisely, for any J C N and any assignment ip: J —> A of 
0’s and l’s to elements of J there is a point x in the full shift realizing this assign¬ 
ment, that is, Xi = ip(i) for all i £ J. For a general shift space X over {0,1} the 
above procedure may not work for every set JcN. But one may still consider the 
following problem: let X be a shift space over A. Assume that someone picks a set 
JcN and for each i £ J chooses a symbol <p{i) £ A (the choice may be random). 
Does there exist a point x £ X whose i-th coordinate is ip(i) for every i £ .7? If 
the answer is positive for every assignment ip, then we say that X is independent 
over J. Now we may “measure” the randomness of X by the size of the “largest” 
independent set J C N. 

Definition (Independence set for a shift space). We say that a set J C N is an 
independence set for a shift space X C if for every function ip : J —> A there is 
a point x = {xj}f^ 1 £ X such that Xi = ip(i) for every i £ J. 

Note that if J C N is an independence set for a shift space X , then so is every 
subset of J. 


5.1. Asymptotic density and Shnirelman density. The most natural way to 
describe the size of an infinite subset of N is the asymptotic density. It is transla¬ 
tion invariant and it is invariant under the exclusion or inclusion of finitely many 
elements. A similar notion is the Shnirelman density , which formally is also a limit, 
but it gives information about the structure of A D {1,..., n) for every n £ N. Al¬ 
though it seems to be less natural a concept than asymptotic density, Shnirelman’s 
notion proved invaluable to his approach to the Goldbach problem. 


Definitions (Densities). A set A C N has asymptotic density a if the limit 


d{A) 


lim 


|An{l,2,...,n}| 


n 
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exists and is equal to a. 

The Shnirelman density cfeh(A) of a set A C N is defined by 


dsh{A) = inf 


1^4 PI {1,2,, n}\ 


: n G N 


5.2. Positive entropy is equivalent to independence. Both notions measure 
how dense a set is, and a set with any of these densities positive may be called 
large. We will prove that positive topological entropy is equivalent to existence of 
an independence set for which asymptotic and Shnirelman densities are equal and 
positive. This shows that for a shift space with positive entropy one can find an 
independence set which is large and structured (there exists 0 < a < 1 such that 
our set occupies at least a proportion of {1,... ,nj for every n G N). Note that 
positive asymptotic density of a set A tells us only that \A fl {1,... ,n}\ behaves 
like an for some a > 0 and n large enough. 


Theorem 1. Let X be a binary shift. Then the entropy of X is positive if and only 
if X is independent over a set whose asymptotic density exists, is positive, and is 
equal to its Shnirelman density. 

This is a strengthening of (4Ql Theorem 8.1]. A general result applicable to all 
dynamical systems was proved by Glasner and Weiss [15] and Huang and Ye El 
Theorem 7.3]. Kerr and Li extended it further in [JO]- We add the Shnirelman den¬ 
sity to the picture, which shows that the independence set is even more structured. 
A similar result holds for more numerous alphabets (cf. [40], Theorem 8.3]). We 
state it without a proof. 


Theorem 2. Let r > 2 and X be a shift space over A = {0,1, ..., r — 1}. Then 
h(X) > log(r — 1) if and only if X is independent over a set A whose asymptotic 
density exists, is positive, and is equal to its Shnirelman density. 


Before the proof we introduce all the necessary tools. 


5.3. Limiting frequency. Our main technical tool is the limiting frequency. 


Definition (Limiting frequency). Let A. be a finite alphabet and X be a shift space 
over A. For every symbol a G A and every point x G X we define the characteristic 
set Xa{x) of a in x as the set of positions at which a appears in x, that is, 

Xa{x) = {j G N : Xj = a}. 

Let w = w\... Wk and let ||-u;|| a denote the number of a’s in w, that is 

IMU = K 1 <j<k: Wj = a}|. 

Let MJl(A') be the maximal number of occurrences of the symbol a among all blocks 
w G $k(X), that is, 


MJl(Al) = max{||u;|| 0 : wGT> k {X)j. 


The sequence {M^(X)}^ =1 is non-negative and subadditive, that is 


0 < M^ n+n (X) < M(^(X) + M“(Y) 

for any positive integers m and n. By Fekete’s Lemma the sequence {M k (X)/k}^L 1 
converges to its greatest lower bound. We call this limit the limiting frequency of 
a in X and denote it by 


Fr 0 (X) 


lim 

k—too 


M a k (X) 


inf 

k> 1 


MgPQ 

k 


( 2 ) 


k 
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It can be shown that a limiting frequency of a symbol a is precisely the maximum 
of measures of the cylinder of a with respect to ergodic invariant measures supported 
on X. The next theorem is a strengthening of a well-known result about the 
ordinary density. It also follows^ from the Ruzsa Theorem [33] Theorem 4, p. 323] 
(see also [30]). 

Theorem 3. Let X be a shift space over an alphabet A. Then for every symbol 
a £ A there exists a point oj a £ X such that 

dsh{Xa{u a )) = d(xa{u a )) = Fr a (X). 

Proof. Clearly, Fr a (X) = 0 implies dsh(Xa(x)) = d(x a (x)) = 0 for all x £ X. We 
may therefore assume that Fr a (X) > 0. 

For every m > 0 let w^ = w^ .. .wffl £ ' H m (X ) be a block which attains 
the maximal number of occurrences of the symbol a, that is, 

lM m) IU = M^(X) = max{||«;|| a : w £ , B m (X)}. 

We claim that for each integer k > 0 there exists a block £ Hk(X) such 
that for all 1 < j < k we have 

(3) j- Fr a (X)<||4 fc) ...«;f ) || a . 

In other words, we claim that given k > 0 we can find a word of length k such that 
for each 1 < j < k the average number of occurrences of a in the prefix w[ k> ... 
of w is not smaller than the limiting frequency of a. 

For the proof of the claim, assume on the contrary that ([3]) does not hold for 
some k > 0. This means that every block w of length greater than or equal to k 
has a prefix of length 1 < j < k for which © fails, that is, such that 


ll"'i •••“■/! 


< Fr a (X) < 1. 


Note that the left hand side of the above inequality is always a fraction of the form 
6/c, where 0 < b < c < k. Since k is fixed, there are only finitely many such 
fractions, hence the number ao = min{Fr a (X) — 6/c>0:0<6<c<fc}is 
well-defined (the minimum exists) and is positive. 

Take a positive integer N such that L-^ a oJ = 1- Since we assumed that our 
claim fails, and since p = k 2 N + 1, we can chop the block w ^ defined above into 
at least Nk +1 pieces, each of length k at most, and for all but at most one of them 
© fails. To see it, we can imagine that we apply to the block the following 
procedure: for the first step, we set 1(0) = 0. Then we find the smallest j such 
that © fails for w^ p \ By our assumption there is such j , moreover, j < k. We set 
Z(l) = j and define 

1, 


ai = Fr a (X) - 


(P)„7,(P) 


\w] w, 


J 


■ w^l 


Clearly «o < oq. For the next step we consider 

-(p) -(p) 

V = W J+ 1 W j+2 ...l 

that is, we chop off the prefix of length j from uf p \ We proceed inductively. 
Assume that we have performed s steps. This means that 1(0) < ... < l(s) and 


r,(p) 

U P ? 


2 We are indebeted to Vitaly Bergelson for pointing this out. 
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a(l),..., a(s) are defined for some s > 1. If p — l(s) < k, then we set l(s + 1) = p 
and we finish the construction. Otherwise, we have p — l(s) > k and we consider 


v _ , 7 ,(p) „t,(p) „T,(P) 

V W l(s)+l W l(s)+2 ' ■ ' W p ' 


Then we find the smallest j such that ([3| fails for v . We set l(s + 1) = l(s) +j and 


Or s +l 


Fr a (X) - 


~(P) 


l(s + l) — l(s) IHsl+OO+i)] 


> ao > 0. 


Let t denote the number of steps we have performed. We have found a strictly 
increasing sequence of integers {7(s)}*jlo as we ^ as a sequence of positive real num¬ 
bers {a s }* =1 such that 1 (0) = 0, l(t + 1) = p, for every j = 1,2 ..., t + 1 we have 
0 < l(j) — l(j — 1) < k, and 


(4) ll*i8_ 1) + llIW)] ||a = m - l(j - m^a(X) - aj) < 

(' l(j) -HJ- i))(Fr a (X) -a 0 ) for j = 1 

Note that it may happen that l(t) = l(t + l) = p , but the inequality ||w|/^ +1 p j|| a < 

k is valid regardless (if l(t) + 1 > p, then we agree to identify p , with the 

empty block). Clearly, we have t > kN, hence by the inequality in j4]) and the 
definition of N we have 

ll w(P) ||a = + H W [dt) + l,p]H a 

S=1 7 

< p(Fr a (X) - a 0 ) + k < pFi a (X) - k 2 Na 0 + k < pFr a (X). 

Now we have the required contradiction, since should be greater or equal 

to pFr a (X). 

We proved our claim is true, hence we have a sequence of blocks with 

w ( k ) from T>k(X), satisfying fl3j. We will usc0 the to find a point x € X such 
that for each k > 0 the block army is a prefix of some with l(k) > k. 

For k £ N we inductively find a decreasing sequence Sk of subsets of N so that for 
any n £ Sk with n > k all prefixes are equal. We begin by picking a symbol 

a £ A so that w[ k ^ = a for infinitely many k's. Let Si = {k € N : = a}. 

Next, since the set of possible blocks of length 2 in X is finite, there is an infinite 
subset S 2 C Si such that the prefix wC ^ is the same for all n £ S 2 . Continuing 
this way, we find for each k > 2 and infinite set Sk C Sk -1 so that all blocks 
are equal for n £ Sk- Define x to be the sequence with Xk = w^ for some n £ Sk 
with n > k. Note that then arji y = ui^l, k\ for every n £ Sk \ {1,..., k — 1} since 
Sk C Sk -1 for k > 2. Therefore for each k > 0 the block arny is a prefix of some 
w (K k )) w ith l(k) > k. In particular, arny £ 'B(X) and hence x £ X. 

For every k £ N we have 

(5) Fr^X^ill^yll^M^pO, 


^This can be obtained easily by using a standard compactness argument, but the referee has 
encouraged us to present a completely elementary proof of this fact. 
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where the first inequality above is a consequence of the claim and the construction 
of the point x, while the second inequality follows from the definition of M^(X). 
Taking the inhma over all k G N of all terms in © and applying © we obtain 

Fr a (X) < inf i||x [ljfc] || a = d Sh ({j G N : xj = a}) < inf^Mj(I) =Fr a (X). 

Since the values M k a {X)/k converge to Fr a (X) as k goes to infinity (see ©) we 
conclude that 

lim 7 ||xr life] || a = d({j G N :Xj= a}) = d Sh ({j € N : Xj = a}) = Fr a (X). □ 

k—Voo K 

5.4. Some combinatorial Lemmas. The following lemma is well-known. 

Lemma 1. Let 0 < e < 1/2 and n > 1. Then 



where H (e) = — e log e — (1 — e) log(l — e). 

Definition (Independence sets for blocks). Let 3 be a (possibly empty) family of 
binary blocks of length n > 0. We say that 3" is independent over a set JcN and 
J is an independence set for 3" if for each map ip: J -A {0,1} there is a block w G 3 
whose i-th symbol is ip(i) for every i G J. 

We denote the collection of all sets of independence for 3 by 3(3'). We assume 
the convention that the empty set is a set of independence for every (including 
empty) family of n-blocks. 

We will need a variant of the famous Sauer-Perles-Shclah Lemma (Lemma [3] 
below, see [33], SB]). The proof follows [2], where the next lemma (stated for sets) 
is attributed to Alain Pajor [2B1 . 

Lemma 2. Let 3 be a family of binary blocks of length n > 0. Then | 3(3) | > |3|. 

Proof. The proof goes by induction on n. For n = 0 the situation is clear: 3 C {e}, 
hence (3^ < 1 and 3(3 r ) > 1 because of the empty set. 

Assume that the lemma holds for all families of n-blocks for some n > 0. Consider 
a family C {0,1}" +1 . Let 3/ be the family of prefixes of length n of all blocks in 
3 r ending with 0. Analogously, we define 3i as the family of prefixes of length n of 
all blocks in J ending with 1, that is, 

3/ = {w G {0,1}" : w;0 G 3 r }, 

3 r i = {w> G {0, l} n : wl G T}. 

Since evidently, = IS^I + |3q|, we may apply the inductive assumption to infer 
that our lemma holds whenever IS^ol = 0 or |3 r i| = 0. So assume that |To| ^ 0 and 
|3i| ^ 0. Then every set which is independent for 3 r 0 or 3q is also independent 
for 3\ Unfortunately, there are sets which are independent for both 3/ and 3q 
(the empty set, for example). However, if J G 3(3o) D 3(34), then J U {n + 1} is 
independent for 3, but J U {n + 1} is neither in 3(3o) nor in 3(3i). Therefore, by 
the inclusion-exclusion formula 


I 3(3)| > | 3(3 0 ) U 3(30| + I J(J'o) n 3(3,)| = | 3(3 0 )| + | 3(3 X )|. 
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But applying the inductive hypothesis we obtain 

|J(?o)| + |^l)|>|?o| + |?l| = |?| 

and the proof is finished. □ 


Lemma 3. Let IF C {0, l} n be a family of binary blocks of length n > 1. 
some 1 < k < n we have 



then IF is independent over some set of cardinality k. 


If for 


Proof. By the previous lemma and our assumption we have 

k -1 

roi>m>E 

j=0 

As the last sum is the cardinality of the family of all subsets of {1,..., n} with less 
than k elements there must be a set with k elements in 3(3 r ). □ 

As a corollary of Sauer-Perles-Shclah Lemma and Lemma [1] we obtain a variant 
of the Karpovsky-Milman Lemma m- 

Lemma 4. Let X be a binary shift with positive topological entropy. Then there is 
an e > 0 such that for every n > 1 there is a set J C {1,..., n} with [enj elements 
which is an independence set for X. 



Proof. Take 0 < e < 1/2 such that 0 < H(s) < h(X ), where H{e) = —eloge— (1 — 
e) log(l — e). By Lemma Q] we have 

\ne\ . . 

E f 1 < 2 n ' ff(E) < 2 nh ( x ' ) < |'B„(X)|, 

l=o 

for any n > 1. It follows from Lemma [3] that there is a set of independence for 
l B n (X) with [sn\ elements. □ 

5.5. Proof of Theorem [TJ 


Proof of Theorem\J\ Let X be a binary shift. 

Assume first that X has positive entropy. Let C n be the collection of charac¬ 
teristic functions of sets of independence for ’B n (X). We can treat each element 
of C n as a binary block; then C = (J/L , is a factorial and prolongable binary 
language. We denote the shift space it defines by lx ■ 

By compactness of the full shift and the fact that a point x £ is in Xc if and 
only if £ C for all i,j £ N with i < j, elements of lx may be identified with 
characteristic functions of independence sets of X. 

It follows from Lemma ED that Fr-L {I x ) > 0. Using Theorem [3] we may now fix 
an element of Ix which is a characteristic function of J, an independence set for 
X, such that dsh(J) = d(J) = Fri(Tjf) > 0. 

If, on the other hand, J is an independence set for X such that dsh(<F) = d(J) > 
<5 > 0, then for every n £ N there are at least n5 elements of J in {1,..., n}, and 
therefore |'B ra (X)| > 2 nS for every n £ N. This implies that h(X) > S and completes 
the proof. □ 
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Remark. It follows from the above proof of Theorem Q] that for every binary shift 
X one has 

h{X) > Pn (I x ). 

One may wonder, whether the above inequality is, in fact, an equality. It holds, for 
example, for the square-free flow S (see [.'1 11). Unfortunately, this is not always true. 
Consider the golden mean shift G (the shift defined by taking T = {11} as the set of 
forbidden blocks). Note that it is hereditar^, that is, given a word in its language, 
one can replace any number of l’s by 0’s and the resulting word will still belong to 
the language. Therefore Ig = G. It is known that h{G) = log((l + v / 5)/2), but it 
is easy to see that Fri (Ig) = 1/2. 

6. Entropy, Topological Mixing, Ergodic Measures, and Chaos 

In this section we construct an example of a shift space with the same properties 
as the symbolic system constructed in [39] by Weiss. We begin by defining induc¬ 
tively a sequence of shift spaces Xk so that Xo C X\ C ... and these inclusions are 
strict. After completing the induction we define 

OO 

x = (J x n . 

n =0 

This means that 23(X) = 'B(Xq) U 23(Xi) U 23(AC 2 ) U .... It will be clear that at 
each step of our inductive construction some words are added to 23 (X n ) to form 
'B(X n+ 1 ). We say that a block u £ {0,1}* with ||u||i > 0 is added at step n £ N if 
u £ 'B(Xn) \ 'B(Xn-i). Any block u £ 23(AT 0 ) with ||u||i > 0 is added at step 0 by 
definition. Recall that ||u>||i denotes the number of l’s in a word w. A word w is 
a from-l-to-1 word if w = 1, or w = lvl for some v £ {0,1}*. Let Y be a shift space 
over {0,1}. Let 9 n (Y) (S*(T)) denote the set of all from-l-to-1 words in Y with 
at most (exactly) n occurrences of 1. The following two properties of blocks added 
at step n are easy consequences of the definition of X n presented below. They will 
be proved along the lines of the definition. 

Lemma 5. If a block u is added at step n > 1, then there is a prefix of u of the 
form 0 a u'0^u" for some u',u" £ S 2 n-i(X n _ 1 ), a > 0 and /3 > 2 2n ~ 2 . It follows 
that: 

(1) 2 71-1 < |M| i; 

|M|i 2 n _ 1 

| u | - 2 n + 2 2n-2 “ 1 + 2 71-2 ' 

6.1. Construction of X. We inductively construct an increasing sequence of shift 
spaces Xo C X\ C ... and then define X as the closure of the union of all X n ’s. 

Outline of the construction. Recall that a shift space X is mixing if for every blocks 
u, v £ H(X) there is an N £ N such that for each n > N one can find a block w 
of length n such that uwv £ ®(2f). We call such a word w the n-transition block 
from u to v. The periodic points are dense in a shift space X is for every block 
u £ 23 (X) there is a block v such that (uv)°° £ X. If x £ X is a periodic point, 
then the prime period of x is the length of the shortest block w £ 23 (X) satisfying 
x = w°°. 


^Hereditary shifts were introduced in m and examined in m- 
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In our construction the n-transition block can be always a block of 0’s. The 
length N depends on the number of l’s which occur in u and v , namely we choose 
N such that the relative density of l’s in uO N v is small. The periodic points are 
dense in our shift space but the density of l’s in a periodic point decreases very 
fast when the prime period of the point grows. We achieve this by adding periodic 
points of the form x = (rtO n )°° with n large enough to force the relative frequency 
of l’s in a; to be small. 

Definition of Xo . We define Xo to be the set of all sequences in the full shift over 
{0,1} with at most one appearance of the symbol 1. That is, Xo = {0 oo }U{0 a 10 o ° : 
a > 0}. Note that 

®(X 0 ) = {0“ : a > 0} U {0“10 /3 :a,/3> 0}. 

Before we present the details in full generality let us first work out the special case 
of Xi. 

Definition of Xi. We add to Xo the orbits of points of the form: 

(Kb 3 ) 00 , or 0“10 / 3 10°°, where /3 > 1, a > 0. 

Note that every block u added at step 1 has at least two occurrences of the symbol 
1 , and fulfills 

. .. . 2 
IMIi/M < 3 - 

The bound above is the best possible, since it is attained by the block 101 added 
at step 1. Therefore all assertions of Lemma [5] hold. 

Definition of X n+ i. For the inductive step, given X n and n > 0, we construct 
a shift space Y n+ 1 and set X n+ \ = X n U Y n+ \. To define Y n+ \ we specify a set of 
auxiliary points and then let Y n+ 1 be the closure of the set of auxiliary points. 

There will be two types of auxiliary points. We call them periodic and joining 
auxiliary points. Let P ^ +1 be the set of all periodic points of the form (u0 fc )°°, 
where k > 2 2 ” and u £ 92 n {X n ). A point a: is a periodic auxiliary point if it 
belongs to the orbit of some point in P' L+i . We denote the set of auxiliary periodic 
points by P n +i- We have 

OO 

Pn+1 = U ^({^0 fc )°° : k > 2 2ra , u £ S2»(*n)}). 

1=0 

It is clear that P n +i is shift invariant. The set of joining auxiliary points is given 
by J n+ i = J' n+1 U J" +1 , where 

J'n+i = {0 a u0°° : a > 0, u £ g 2 n(X n )}, 

J"+i = {0 a u0Pv 0°° : a > 0, /3 > 2 2n , u, v £ 3 2 n(X„)}. 

Points from guarantee that the mixing condition holds for pairs of blocks u , v 
with at most 2 n occurrences of 1. But J" +1 is not shift invariant and orbits of 
points from J " +1 end up in J’ n+1 U {0 00 } after a finite number of shift operations. 
We claim that 

Pn +1 £ Pn +1 bl Jn+ 1 , 

Jn +1 = ^n +1 U { 0 °°}. 


( 6 ) 

(7) 
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For the proof of the inclusion in (@ assume that a sequence {a ;( m )}^ =1 C P n + 1 
converges to x. Without loss of generality we can assume that 


T (m) _ Am) 




where u N) £ S 2 n (X n ) is a block of length £(m), 1 < t(m) < ell(m) and j3 m > 2 2m . 
We also assume that 0^ m is primitive, that is, it is not a concatenation of two 
or more copies of some block over { 0 , 1 }. 

There are two possibilities: 

Case 1: 9 *„ +1 (%) ^ 0- Let N be the smallest integer such that xnjvj £ S 2 n+i (a;). 
But — > x as m —> oo, thus for all sufficiently large m we have aML = rc[i ; jv]- 

In particular, ar^jv] £ 23 (aM)). Consequently, vS m is a subblock of ir^jv]- 
There are only finitely many subblock of arnjv] of the form u0 k where k >2 m and 
u £ S 2 n {X n )- Therefore there are only finitely many y £ P n +i such that S^n+iiy) 
contains rcri ^v] • It follows that x N) is eventually constant, hence x £ P n + 1 - 
Case 2: S*„ + i (x) = 0. It is easy to see that in this case x £ J ra +i- 
The proof of J7J follows the same lines as the proof above. We leave the details 
to the reader. 

We set 

In+i = Pn +1 U Jn+i and = X n U Y n+1 . 

Clearly, X n C X n+ \. This completes the induction step. Observe that any block 
u added at step n + 1 has a prefix 0 a u'0 l3 u" for some u',u" £ S 2 n (Wi), a > 0 , 
/3 > 2 2 ". Furthermore, u fulfills ||u||i > 2 n as otherwise u would be added at earlier 
step. Therefore 

Mli IKHi + IK'lli 

|u| _ \u'\ + |u"| + /3 ’ 

But ||u , ||i and ||u ,, ||i are bounded by 2™, and we conclude that 


Nli 


< 


2 n+1 


1 


2 n+1 + 2 2n 1 + 2 n ~ 1 ’ 


as claimed in Lemma [5] We note the following consequence of the formulas 0 and 

di¬ 


corollary 1. If x £ X n and the set { j £ N : Xj = 1} is infinite, then x must be a 
periodic point. 


6.2. Properties of X. We describe the properties of X as a sequence of lemmas. 
Lemma 6. Periodic points are dense in X. 

Proof. Take any u £ 23(X). Assume that u is added at step n and let k = ||u||i. 
If n = k = 0, then 0°° is a periodic point in the cylinder of u. If k > 0, then we 
write u = for some u £ S k(X), a, > 0. We set m = [log 2 k~\ and note that 

m > n. Hence u £ S k(X n ) C S 2 m (X m ). We may infer that 

m a+ P +2 J = (u0 2 )°°€P m+ i, 

and therefore there is a periodic point of X in the cylinder of u. □ 


Lemma 7. The shift space X is topologically mixing. 
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Proof. Let u and v be two nonempty blocks in 23(X). Assume that ||u||i > 0 and 
|H|i > 0. Consequently, there is a k > 0 such that u = 0 a u0 6 and v = 0 c u0 d 
for some a,b,c,d > 0 and u, v £ S 2 k (Xk). Therefore uO^vO 00 £ J n +i C X for all 
f3 > 2 2fc . If for some aeNwe have u = 0“ or v = 0“, then our reasoning is similar. 
Therefore X is topologically mixing. □ 

Lemma 8. If x £ X, then either x is periodic or the asymptotic density of the set 
Xi(ie) = {j £ N : Xj = 1} equals zero. 

Proof. Assume that the set Xi ( x ) is infinite. Denote by A„ the step at which 
X[i. n ] was added. The sequence A n is nondecreasing, hence it is either bounded, or 
A n oo. In the former case x £ X m for some m£N, hence x must be periodic. 
In the later case we have 

1(1 < j < n : Xj = 1}[ 1 

n ~ 1 + 2 A " -2 ’ 

which monotonically tends to zero as n goes to infinity by Lemma 0 Cl 

Lemma 9. Every point x £ X is a generic point for some a-invariant ergodic 
measure p x on X and all ergodic invariant measures of ( X , a) are supported on 
single periodic points. 

Proof. If a; is a periodic point, then it is clear that x is generic. If x is not a peri¬ 
odic point, then for each k £ N the asymptotic density of the set Xi( x ) = {j £ N : 
x \j.j+k) = 0 fc } equals one. It follows that x is generic for the Dirac measure concen¬ 
trated on the fixed point 0°°. It follows that there are no other ergodic invariant 
measures. □ 

Lemma 10. The shift space X has no DCS pair. 

Proof. Fix k £ N and take any x, y £ X. We define 

Eq k {x,y) = {j £ N : x [jj+k) = V\j,j+k)}- 

It is easy to see that in the case of symbolic dynamical systems equipped in the 
metric p defined by m for each k £ N and t £ (2 fc , 2 fc+1 ] we have 

(8) F xy (t) = d(Eq k (x,y)) and F* y {t) = d(Eq k (x, y)). 

We will show that Eq fe (x, y) has a well defined asymptotic density. Let Uj = a'yj+k) 
and Vj = y\j,j+k ) f° r j € N. If a: and y are periodic, then obviously Eq k (x,y) has 
an asymptotic density for each k. For the remaining case we assume, without loss 
of generality, that x is generic for the Dirac mass at 0°°. Observe that 

Eq fe (x, y) = {j £ N : Uj = vj} 

= {j £ N : Uj = 0 k = Vj} U {j £ N : Uj = Vj ^ 0 fc }. 

The set {j £ N : Uj = Vj ^ 0 fc } is contained in {j £ N : Uj 0 fe }, hence 
has asymptotic density zero. We also know that d({j £ N : Uj = 0 fc }) = 1 and 
d({j £ N : Vj = 0 fe }) exists, therefore Eq k (x,y ) has a well defined asymptotic 
density. Since k was arbitrary, we apply (jSJ) to conclude that F xy {t) = F* (t) for 
each teK. □ 

The following result may be proved directly, or using a result of Downarowicz 
uni, who showed that any positive entropy system has many DC2-pairs. 
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Lemma 11. The shift space X has topological entropy zero. 

Proof. Note that all ergodic measures of X are concentrated on periodic points, 
hence their measure-theoretic entropy is equal zero. By the variational principle 
the topological entropy is also equal zero. □ 

Finally, we note the following corollary of Lemma [9] 

Corollary 2. There is no ergodic invariant measure with support X. 

Lemma 12. The shift space X is not u>*-chaotic. 

Proof. The only minimal sets in X are periodic orbits. Therefore no w-limit set 
contains an infinite minimal set. □ 

The following lemma is well-known (see [21] Chap. 4, Problem 41]). 

Lemma 13. There exists an uncountable set T and a family {13(7)} 7 gr of infinite 
subsets of N such that for any a, /3 £ T with a ^ (3 we have 

(1) B(a) fl B(/3) is infinite, 

(2) B(a)\B(/3 ) is infinite. 

We now proceed to our last result. 

Lemma 14. The shift space X is u-chaotic. 

Proof. We first define for any infinite set S C N an infinite transitive subsystem 
Xs C X. To do this we proceed as with the construction of X. We start with 
Xq = Xq. During the induction step, we define the set of joining auxiliary points 
by Jn +1 = ^n+i U Jjf+i, where 

Ju+i = {0 a «0°° : a > 0, u £ S 2 » (*£)}, 

JS +1 = {0“u0 /3 u0°° : a > 0, /3 > 2 2n , /3 € S, u,v e S 2 »(X;f)}. 

Note that C J n + 1 and J^+i nonempty because S is infinite. The main dif¬ 
ference from the previous construction is the condition f3 € S. We do not add any 
periodic points. As above we set Xf +1 = X„. U Jn+v Th en 


= U X n-■ 

n =0 

We inductively define the family of allowed blocks. We say that every block 
w £ {0,1}* with ||u>||i < 1 is allowed. Assume that for some n £ N we have defined 
which blocks w over {0,1} with at most 2 n ~ 1 ones are allowed. We say that a block 
w £ {0,1}* with 2 n ~ 1 < 111(1111 < 2 n is allowed if there are allowed from-l-to-1 
blocks u\ and u 2 with 1 < ||uj||i < 2 n ~ 1 for j = 1,2, a,b > 0, and (3 £ 5 such 
that (3 > 2 2n and w = 0 a ui0^it20 b . A straightforward (but tedious) analysis of the 
definition of Xs shows that block w belongs to the language 23(Xg) of Xg if and 
only if it is an allowed block. This finishes the construction of Xg. 

We claim that Xg is transitive. To this and take two allowed blocks u and v. 
Without loss of generality we may assume that u and v are from-l-to-1 blocks. Let 
i = ||u||i and j = ||ti||i. Pick N such that maxji, j} < 2 N . Let f3 £ S be such 
that f3 > 2 2N . Then uQd v is an allowed word, which proves the transitivity of Xg. 
Again, we used the fact that S is infinite. It follows that Xg is a topologically 
transitive and infinite shift space. Therefore Xg is uncountable. 
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Note that if S and T are infinite subsets of N such that S D T is also infinite, 
then XsnT C Xs (~l Xt (here one can write an equality, but this inclusion suffices 
for our purposes). It is also clear that if S' C T C N is infinite, then Xs C Xt- 
Furthermore, if S, T C N are infinite and disjoint, then Xg D Xt = Xq. To see 
this, observe that Xq is contained in Xg for every infinite S. On the other hand, 
if x G (Xg 0 Xt) \ Xo, then we can find w = lO^l G 23(Xs) 0 'B(Xt), which is 
impossible since then p G S 0 T = 0. The same argument shows that if S ^ T, 
then Xg ^ Xt- 

Recall that Xo is a countable shift space. In particular, if S, T C N are infinite 
and disjoint, then Xg \ Xt = Xg \ X 0 is uncountable. 

Let {R( 7 )} 76 r be a family provided by Lemma fTT?l For any a, P G F with a ^ p 
we define C( a, p) = B{a) 0 B{0) and D(a, p) = B(a) \ B(p). 

Let X 7 be the shift space Xs constructed for S = B( 7 ), where 7 G T. Using 
transitivity, for any 7 G F we pick a point x 7 G X 7 such that its omega-limit 
set, bo(Xj) = X 7 . We claim that {ir 7 } 7e r is an w-chaotic set. Clearly, {a; 7 } 7g r is 
uncountable, because X 7 ^ X 7 / if 7 ^ 7 '. Furthermore, B("f) is infinite, therefore 
u(xj) is uncountable for each 7 G T and is not contained in the set of periodic 
points. Now note that for every a,P G F with a ^ P we have uj(x a ) flu(ij) D 
Xc(a,t 3 ), which is uncountable because C(a,P) is infinite and ui(x a ) \ui(xp) is also 
uncountable (it contains all transitive points Xjj^j 3 ), because D(a,P) is infinite 
and disjoint with B{p)). Hence {x 7 : 7 G T} is an w-chaotic set, which proves that 
X is w-chaotic. □ 
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